A Measure of Robustness to Misspecification

Suppose a researcher wants to predict the causal effect of a new policy or treatment, based on a data set containing information on some characteristics and outcomes for a large set of units. The researcher might specify a statistical model relating the outcomes, treatments, and characteristics, estimate that model, and use the results to predict the effect of the policy or treatment. The researcher typically then reports the point estimate of the policy effect as well as a measure of the uncertainty surrounding that point estimate (e.g. the standard error). Such measures of uncertainty depend on the model specification. Often, however, there is also substantial uncertainty regarding the specification of the model. There are typically elements of the specification that are relatively ad hoc, and different choices for those specification decisions would have left to different point estimates of the effect of the policy. Previously, Leamer (1982, 1983) argues that the credibility of much empirical work is very low partly because of specification searches that underly the estimates for the reported models. He proposes reporting estimates based on a larger class of models for a range of prior distributions. White (2000) proposes codifying the model searching process and reporting standard errors that take into account this process. However, it is not standard practice to follow these proposals. In practice researchers typically address the issue by reporting estimates for a small number of different specifications to assess the sensitivity of the estimates to